Multiframe selective information fusion for 'looking through the woods'

A multiframe selective information fusion technique, based on a time evolution equation with anisotropic gain, is applied to the spatio-temporal processing of a video captured from behind visual obstructions, achieving as final output a synthetic frame of unobstructed field of view, as though 'looking straight through the woods' at a distant scene. I. INTRODUCTION The goal of multiframe selective information fusion is to synthesize a montage of focused regions captured in multiple frames into a single contiguous synthetic frame[1,2]. For instance, a video of a distant scene captured by a moving camera from behind highly blurred visual obstacles can be synthesized into an unobstructed field of view, as in the case of 'looking straight through the woods' at a distant scene, that could prove to be of use in surveillance. Two distinct issues must be addressed in the development of the technique. Foremost is the accurate alignment of the multiple frames in relation to the synthetic output frame. With multiple cameras, or with a single camera in slow motion, a scene is captured from different viewpoints in relation to the obstructions. Each captured frame will therefore be globally shifted relative to one another. By computing the correlation function between two images, the inter-frame shift may be determined from the coordinates of the correlation peak. The precision of the correlation method can be enhanced by using edge maps instead of the actual images, where an edge map is defined to be either the gradient or Laplacian of an image. In the second stage of processing, the focused regions within each input frame are extracted and fused into the synthetic output frame. This is accomplished by a